Integrated virtual patient framework

ABSTRACT

An Integrated Virtual Patient Framework (IVPF) for providing decision support that evolves with increasing data collected on a given patient. Support decisions for patients with limited data is made using statistical prediction tools derived from historical trajectories of similar patients. As patient histories grow, decision support is provided by mathematical models that constrain the possible dynamics of the patient to more detailed predictive models. Weights for each model are assigned depending on uncertainties arising from data fitting and model properties. As new data are entered into a patient record, the models are recalibrated and the weights are adjusted, leading to updated decision support information. The framework also suggests the benefit of additional follow-up data collection events, optimizing the data collection as well as how the IVPF generates predictions. The framework also may present scenarios to patients in a way that informs them of treatment outcomes, given various strategies.

CROSS-REFERENCED TO RELATED APPLICATIONS

This application in a continuation-in-part of U.S. patent applicationSer. No. 15/032,969, filed Apr. 28, 2016, entitled, INTEGRATED VIRTUALPATIENT FRAMEWORK, which is a U.S. National Stage filing under 35 U.S.C.§ 371 of PCT/US2014/63341, filed Oct. 31, 2014, which claims the benefitof priority to U.S. Provisional Patent Application No. 61/898,990, filedNov. 1, 2013.

BACKGROUND

Conventional applications used in the clinic to inform treatmentdecisions are typically limited to a single data time point, they arestatistically derived, and they accept only limited patient-specificdata. These data (i.e., age, tumor grade, tumor size, lymphaticdissemination, etc.) are used to subdivide the entire cohort of patientsin the historical record into a sub-cohort that has similar propertiesas those entered by the clinician. The software then compares outcomesof this sub-cohort according to the treatment they received.

However, these applications have several limitations. First, they canonly subdivide patients across parameters which have been measured andrecorded in the historical database. Second, they can only give resultsfor therapies which have been used historically on significant numbersof patients. Third, there is no method to use temporal patient-specificdata to refine the predicted outcomes.

SUMMARY

The present disclosure describes an Integrated Virtual Patient Framework(IVPF), which is an architecture for optimizing patient-specificclinical decisions that are simulated by mathematical model modules,accomplished directly through a clinical software application. The IVPFprovides decision support that evolves with increasing data collected ona given patient by refining weights and models used to determinerecommended therapies. The framework also may present scenarios topatients in a way that informs them of treatment outcomes, given variousstrategies.

In accordance with an aspect of the disclosure, a model-based method fortherapeutic decision support in an Integrated Virtual Patient Framework(IVPF) is described. The method includes defining a firstdisease-specific model, and one or more additional disease specificmodels that provide therapeutic decision support in accordance with atemporal relationship with multiple timepoints after a patient isdiagnosed with a condition; determining, in accordance with each set ofpatient data at a given time point, a first therapeutic decision byapplying a first weight to the first disease-specific model, a secondweight to the second disease-specific model, and subsequent weights tothe additional disease-specific models; and ranking therapy choices ofthe first therapeutic decision in a user interface. Thereafter forsubsequent patient data received by the IVPF, the following is repeated:adjusting the first weight, second weight and third weight in accordancewith the types of subsequent patient data received; comparing thesubsequent patient data to virtual patent data in a database ofsimulated outcomes determined using the first disease-specific model,the second disease-specific model, and the third disease-specific model;determining, in accordance with the comparing, a subsequent therapeuticdecision by applying an adjusted first weight to the firstdisease-specific model, an adjusted second weight to the seconddisease-specific model, and an adjusted third weight to the thirddisease-specific model; and ranking subsequent therapy choices of thesubsequent therapeutic decision in the user interface.

In accordance with another aspect of the disclosure, a model-basedmethod for therapeutic decision support in an Integrated Virtual PatientFramework (IVPF) is disclosed. The method includes receiving, at a firsttime, first data associated with a patient; defining at least twodisease-specific models that provide the therapeutic decision support inaccordance with a temporal relationship with the first time and a secondtime; determining an initial therapeutic decision by applying weights toa first disease-specific model of the at least two disease-specificmodels; receiving, at a second time later than the first time, seconddata associated with the patient; and determining a subsequenttherapeutic decision by: comparing the first data and the second data tovirtual patients stored in a database of simulations performed in usingthe at least two disease-specific models; applying weights to thevirtual patients to generate a cloud of similar virtual patients to thepatient for each of the at least two disease-specific models; andsubjecting the similar virtual patients to a trial process usingavailable treatments to determine a range of outcomes; and rankingsubsequent therapeutic decisions associated with the outcomes in a userinterface.

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the detaileddescription. This summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

Many aspects of the disclosure can be better understood with referenceto the following drawings. The components in the drawings are notnecessarily to scale, emphasis instead being placed upon clearlyillustrating the principles of the present disclosure. Moreover, in thedrawings, like reference numerals designate corresponding partsthroughout the several views:

FIG. 1 illustrates a framework to validate outcome predictions of asimulation module against historical data;

FIG. 2 illustrates a framework for use a validated module to populate avirtual patient database of optimal clinical outcomes;

FIG. 3 illustrates a framework for performing an initial clinicaldiagnosis and therapy optimization;

FIG. 4 illustrates a framework for prospective patient tracking anddynamic therapy optimization;

FIG. 5 is a schematic block diagram of the components of an IVPFenvironment;

FIG. 6 represents a high-resolution output of the clinical outcomepredicted by the module, with a single patient-specific parameter;

FIG. 7 illustrates a cross section of the output;

FIG. 8 illustrates the results of a user input;

FIG. 9 shows a schematic using databases in combination with thepatient-specific virtual cohorts to determine dynamically optimizedtreatment strategies;

FIG. 10 represents the outcome of running a dynamic treatmentoptimization on a patient with two measured clinical parameters, and twotreatment control parameters;

FIGS. 11 and 12 illustrate user interfaces of a clinical application;

FIG. 13 shows a time of patient data and the evolution of models fromstatistical models to dynamic models;

FIG. 14 illustrates an example workflow of the evolution of the modelsin accordance with patient data; and

FIG. 15 shows an example computing environment.

DETAILED DESCRIPTION

Overview

The Integrated Virtual Patient Framework (IVPF) of the presentdisclosure incorporates dynamic and mechanistic modeling to provide fortesting of finer patient-specific data subdivisions, and also allowsnon-standard therapies to be queried for success. In addition, newmeasurements of patient follow-up data can be rapidly incorporated intothe IVPF in order to dynamically update the optimization of thetreatment strategy, making the IVPF a powerful tool for implementingadaptive therapies.

Several features of the framework will now be described. The software isaccessible to the non-mathematician. This means that inputs, options,and decision recommendations are delivered in a fashion that will haveclear meaning to the clinician deciding the treatment. The system isadaptable to the different decision processes which are used in theclinic. These may include discrete decisions (i.e. treat or don't treat;choice between a number of fixed therapy options), continuous decisions(i.e. dosing, scheduling, duration), and hybrid decisions (i.e.combinations of discrete and continuous decisions). Each disease has aparticular decision set that the framework will be able to handle. Theframework is structured so that the specifics of the biological diseaselie within the swappable mathematical modules. This allows for modulesto be added, updated, and combined, without affecting the generalizedmethods used by the framework to inform the clinical decisions.

Term Definitions

As used herein, the following definitions apply to the following terms:

Clinical decision: The overall decision of how to treat the patient.These are specified by one or more control parameters.

Control Parameters: These are the specific treatment parameters that arecontrollable by the clinician (i.e., type of therapy, dose, duration,etc.).

Optimization criteria: The outcome that is being optimized. Examplesinclude progression-free survival time, curability, drug toxicity, etc.

Historical data: data on a group of patients having a particulardisease, such as breast cancer, and any subdivisions of that data.

Pre-decision data: Patient-specific data collected from a clinicalpatient before the clinical decision is made.

Simulation module (SM): disease specific mathematical model that acceptspatient-specific inputs, control parameters, and delivers a metricrelevant to the optimization criteria

Virtual patient database (VPD): storage for data simulated using themathematical modules. The database has two parts: an optimized outcomedatabase and a temporal simulation database.

Patient-specific virtual cohort (PSVC): The subset of simulations fromthe VP database derived from individual patient data, includingunknown/unmeasured data.

Risk-reward (RR) controls: variables that are controlled by the user inthe software interface to allow for clinician input on the weight ofvarious factors in the optimized results.

Example Workflow

With reference to FIGS. 1-4, in accordance with aspects of the presentdisclosure, the IVPF may operate in four phases: (1) validate themodule, (2) populate the databases, (3) optimize the initial clinicaldecision for individual patients, and (4) prospectively track and refineindividual patient treatment and outcome predictions. The first twophases are performed before the system is used in the clinic. Thisfoundation is then used for rapid initial decision making in Phase 3 andsubsequent patient tracking and dynamic therapy optimization in Phase 4.

A brief description of the phases is given here, followed by additionaldetails.

Phase 1: Module validation. In this phase, the framework is used to testthe predictions of a simulation module developed for the IVPF. Thesesimulated outcomes are compared with historical outcomes for actualpatients.

Phase 2: Module analysis and database population. Once the module isvalidated, the IVPF uses the module to generate a database of outcomesthat can be called upon to determine optimal clinical decisions in Phase3. Temporal data is stored for use in the adaptive therapy of Phase 4.

Phase 3: Initial diagnosis and therapy optimization. A clinician inputspatient-derived pre-decision data into a software application. Theclinician also chooses acceptable levels of risk related to thepatient's potential treatment plan, which can include risk of treatmentfailure, toxicities, patient compliance, co-morbidities, etc., throughthe setting of one or more risk-reward sliders. The IVPF uses thisinformation to parse the outcomes in the VP database in real time andderive predictions for a patient-specific virtual cohort that inform theactual clinical decision.

Phase 4: Prospective patient tracking and dynamic therapy optimization.The IVPF tracks each individual clinical patient by using existingpatient data and the mathematical module(s) to generate detailedpatient-specific temporal outcomes for the therapy chosen in Phase 3. Atthe time when follow-up data is collected (i.e., blood work, imaging,biopsies, toxicity reports, etc.), this temporal data is used to furtherrefine the PSVC of the patient. Additionally, new settings forrisk-reward sliders can be applied given the clinicians objectiveresponse to the therapy to date. These new data and clinician inputswill lead to updated predictions of subsequent optimal therapy.

Prior to the implementation in the IVPF, each simulation module isdeveloped for the particular disease and relevant clinical decision(s).The development of a particular SM is not directly part of the IVPF. TheIVPF does not specify the methods used to model the disease. However,the SM may satisfy the following requirements so that they work withinthe IVPF:

-   -   (i) The SM outlines the range of all inputs and control        variables, and also provides one or more output metrics;    -   (ii) The SM provides information on any additional risk-reward        metrics particular to the disease in question;    -   (iii) For validation, a relevant dataset of outcomes pertaining        to the disease in question is provided, with inputs and outputs        relevant to the SM. In other words, the SM should be directly        comparable to an output metric derived from clinical cohort        studies.

A detailed description of Phases 1-4 will now be provided. Referring nowto FIG. 1, there is shown a framework 100 for Phase 1 of the IVPF. InPhase 1, the IVPF uses a mathematical simulation module 106 to validatethe outcome predictions of the module against historical data 102 andpre-treatment data 104. At The IVPF will call on the module 106 tosimulate the patients in the historical dataset, subject to any measuredpatient data and control parameters. Unknown parameters may be variedthroughout the range accepted by the module. This will produce ahistorical virtual patient cohort (108). The outcomes predicted for thehistoric virtual cohort will be compared to the true historical outcomes(112) in a validation 110. If the validation is not a statisticallyaccurate representation of the actual outcomes observed in thehistorical data, the module would be returned for additional development114. Once a satisfactory validation has been achieved for the module,Phase 1 would be complete and the module would be ready to move to Phase2 (116).

Alternatively, the module 106 could be extended to predict additionalpatient specific parameters which would improve the prediction ofpatient outcomes. This Phase 1 extension would essentially be performedwith additional data collection followed by repeated validation.

An example of how a series of modules would be validated and extended toincorporate additional parameter effects will not be described. In orderto illustrate how the IVPF might be used to predict and validate theeffect of new patient-specific measurements, we have constructed somehistorical data for a generic disease. In this historical patientcohort, the patient-specific parameter p1 is measured as either hi orlow. In addition, there is historical outcome data on these patientssubject to three therapeutic options. The patients were either giventherapy A, therapy B, or no therapy. The outcome metric (i.e., five-yearsurvival) for this historical data is shown in table 1, where a higheroutcome percentage is better.

TABLE 1 Historical true patient data with p1 measurement, for threetherapeutic options Measuring P1 only p1 lo p1 hi Rx A 7.5 39.25 Rx B1.25 46.25 Ctrl 3 20.5

In Table 1, patients with low p1 (left column) have very poor outcomesregardless of the therapeutic approach. Patients with high p1 (rightcolumn) are more responsive to all three therapeutic options. Thehistorical data would suggest that therapy A is the best choice when p1is low, and therapy B is the best choice when p1 is high.

Suppose a mathematical model module is built to simulate the disease andoptimize therapies, labeled Module 1. The module uses three parametersas inputs, p1, p2, and p3, each of which can be either high or low. Thecombinations of these three parameters leads to eight patient types. TheIVPF would begin by simulating these eight types of patients, combinedwith receiving one of the three therapeutic options. This leads totwenty-four outcomes for the patients. These data are shown in Table 2,with each column representing a patient class as delineated by theparameter settings shown in the bottom three rows.

TABLE 2 Simulated patient data using Module 1, for three therapeuticoptions Treatment Model No. 1 Rx A 14 5 22 4 88 65 94 64 Rx B 22 7 25 555 23 43 26 Ctrl 5 2 4 2 21 16 18 12 p1 (meas) lo lo lo lo hi hi hi hip2 (unk) lo lo hi hi lo lo hi hi p3 (unk) lo hi lo hi lo hi lo hi

The data in Table 2 cannot be compared directly to the historical dataof Table 1 because the values of p2 and p3 are not known in thehistorical data. Therefore, the IVPF integrates across the dimensions ofp2 and p3 to derive a comparison dataset from the simulated data. Thiswould generate Table 3. In this simple case, all four data points for p1low in Table 2 for each therapy are averaged, leading to six data pointsin Table 3.

A validation check between the simulated outputs of Table 3 and thehistorical outcomes of Table 1 would show that this first model is not agood prediction tool. The simulation results do not predict the righttherapy for either p1-high or p1-low groups. In addition, itsignificantly overestimates the outcome data for several groups ofpatients. This module would fail the validation step of Phase 1 and bereturned for further development.

TABLE 3 Simulated patient data using Module 1, integrated across p2 andp3 data Model: Measuring P1 only p1 lo p1 hi Rx A 11.25 77.75 Rx B 14.7536.75 Ctrl 3.25 16.75

After further development the new model, Module 2, is submitted to theIVPF. Again the IVPF performs a new validation check as described forModule 1, and generates Table 4.

TABLE 4 Simulated patient data using Module 2 for three therapeuticoptions Treatment Model No. 2 Rx A 24 19 5 4 65 29 48 5 Rx B 1 0 1 5 8651 12 41 Ctrl 1 2 4 2 22 19 8 5 p1 (meas) lo lo lo lo hi hi hi hi p2(unk) lo lo hi hi lo lo hi hi p3 (unk) lo hi lo hi lo hi lo hi

Again the IVPF integrates across the unknown dimensions of parameters p2and p3 to generate Table 5, segregated by patient p1 values.

TABLE 5 Simulated patient data using Module 2, integrated across p2 andp3 data Model: Measuring P1 only p1 lo p1 hi Rx A 13 36.75 Rx B 1.7547.5 Ctrl 2.25 13.5

This module satisfies the validation step, as it predicts the historicaldata of Table 1 with significant accuracy. Module 2 could then be sentforward to Phase 2 of the IVPF for analysis, database population, andeventual clinical use. Here, we use Module 2 to describe the auxiliaryPhase 1.5, in which the validated module is used to predict novelpatient measurements that can further refine the outcome predictions.

By using the full simulation data from Table 4, the IVPF can check tosee which combinations of parameter measurements would give additionaloutcome segregation. By integrating only across p3 and p2, the followingtwo outcome tables in Table 6 can be generated.

TABLE 6(a) Simulated patient data using Module 2, integrated across p3data. Measuring p1 and p2 lo/lo lo/hi hi/lo hi/hi Rx A 21.5 4.5 47 26.5Rx B 0.5 3 68.5 26.5 Ctrl 1.5 3 20.5 6.5

TABLE 6(b) Simulated patient data using Module 2, integrated across p2data. Measuring p1 and P3 lo/lo lo/hi hi/lo hi/hi Rx A 14.3 11.5 56.5 17Rx B 1 2.5 49 46 Ctrl 2.5 2 15 12

The data from Table 6 suggest that measuring p2 would have littleadvantage. For patients with p1-high, the suggested therapy would remaintreatment B, so p2 would not alter the clinical decision. However, panel(b) of Table 6 shows that the measurement of parameter p3 wouldsegregate patients with p1-high into two groups with different optimaltherapy. For p1-high, p3-low patients, therapy A is now preferable totherapy B. Patients with both p1-high and p3-high would do better toreceive therapy B.

In order to validate these results, it would be necessary to collect p3data from patients and observe their outcomes. In some cases, this maybe retrievable from the original dataset, in the case where tissuesamples, gene sequencing, or imaging have been retained but notanalyzed. In other cases, it may involve a prospective study on newpatients. In either case, this new data will generate a more detailedhistorical outcome data. Table 7 shows the expansion of the Table 1 datato account for differences in p3 in patients.

TABLE 7 Historical patient data, measured for p1 and p3 Measuring p1 andP3 lo/lo lo/hi hi/lo hi/hi Rx A 9 6 56 22.5 Rx B 1 1.5 65 27.5 Ctrl 4 235.5 5.5

Unfortunately, the predictions of Module 2 have been disproven by theadditional data collection. The p1-high, p3-low group is still betterwith receiving therapy B, and not therapy A as predicted. Therefore,Module 2 would be rejected for fit and returned for further development.

Finally, Module 3 is developed. In this case, the module produces thedata shown in Table 8. Module 3 can be compared to the historical datafor both p1 and p3 from both Table 1 and Table 7 using similarintegration techniques as before, giving rise to Table 9. In this case,the module satisfies both the historical data for p1 only, and for p1and p3 together, as seen in Table 9, panels (a) and (e) respectively.Furthermore, the module predicts that the measurement of p2 would beuseful for additional patient segregation (panel (d)).

TABLE 8 Simulated patient data using Module 3 for three therapeuticoptions Treatment Model No. 3 Rx A 6 11 12 2 74 37 36 2 Rx B 2 0 3 1 7515 48 37 Ctrl 0 0 0 0 41 12 26 7 p1 (meas) lo lo lo lo hi hi hi hi p2(unk) lo lo hi hi lo lo hi hi p3 (unk) lo hi lo hi lo hi lo hi

TABLE 9(a) Simulated patient data using Module 3, integrated acrosspairs of parameters. Measuring P1 only p1 lo p1 hi Rx A 7.75 37.25 Rx B1.5 43.5 Ctrl 0 21.5

TABLE 9(b) Simulated patient data using Module 3, integrated acrosspairs of parameters. Measuring P2 only p1 lo p1 hi Rx A 32 13 Rx B 22.7522.25 Ctrl 13.25 8.25

TABLE 9(c) Simulated patient data using Module 3, integrated acrosspairs of parameters. Measuring P3 only p1 lo p1 hi Rx A 32 13 Rx B 32 13Ctrl 16.75 4.75

TABLE 9(d) Simulated patient data using Module 3, integrated acrosssingle parameters. Measuring p1 and P2 lo/lo lo/hi hi/lo hi/hi Rx A 8.57 55.5 19 Rx B 1 2 44.5 42.5 Ctrl 0 0 26.5 16.5

TABLE 9(e) Simulated patient data using Module 3, integrated acrosssingle parameters. Measuring p1 and P3 lo/lo lo/hi hi/lo hi/hi Rx A 96.5 55 19.5 Rx B 2.5 0.5 61.5 25.5 Ctrl 0 0 33.5 9.5

TABLE 9(f) Simulated patient data using Module 3, integrated acrosssingle parameters. Measuring p2 and P3 lo/lo lo/hi hi/lo hi/hi Rx A 4024 24 2 Rx B 38.5 7 25.5 19 Ctrl 20.5 6 13 3.5

Once again, additional data collection on p2 values in patients would bederived to check for validation. The historical data would generateTable 10, showing that the model successfully predicts for thesegregation of optima due to p2 status.

TABLE 10 Historical patient data, measured for p1 and p2 Measuring p1and P2 lo/lo lo/hi hi/lo hi/hi Rx A 8.5 6.5 51 27.5 Rx B 1.5 1 45 47.5Ctrl 2 4 25 16

Module 3 would therefore satisfy the validation criteria for parametersp1, p2 and p3 and therefore could proceed to Phases 2 and 3 in order toassist with individual patient-specific clinical decision-making.

This highly simplified example above illustrates the process of usingthe IVPF to predict and validate module outcomes based onpatient-derived data. Though it may seem like the results only reproducethe historical data, this is because the example restricted itself to afew binary parameters and therapies. The actual modules to be used inthe IVPF are likely to include continuous variables for both patientmeasurements and therapy options, and therefore the results will besignificantly more complex. However, the same process can be used forcontinuous variables with this integration and validation approach.

With reference to FIG. 2, there is illustrated a framework for Phase 2(116), where the IVPF will use the validated module 118 to populate avirtual patient database 112 of optimal clinical outcomes. This VPD willcover an entire cohort of virtual patients spanning the complete rangeof possible patient-specific data and clinical controls (120) that areaccepted by the SM. These data will be stored in a way that the IVPF canrapidly aggregate them for delivery of results to the clinical user inPhase 3 (124). Additionally or alternatively, temporal data can bestored in this phase for possible use in Phase 4 (140), depending onstorage capabilities.

With reference to FIG. 3, there is illustrated a framework for Phase 3(124) wherein there is performed an initial clinical diagnosis andtherapy optimization. The outcomes in the VP database 122 developed inPhase 2 (116) may be used by the IVPF to rapidly derive initialpatient-specific recommendations in the clinic. This may beaccomplished, for example, through a clinical application 126 thataccepts patient data 128 and treatment criteria 150. Other interfacesand application may be used to achieve the functions described herein.The IVPF uses the inputs from the clinician to analyze the database andoutcomes (134), smooth the data (136) and generate optimalrecommendations (138) for therapy. The process may then move to Phase 4(140).

As a non-limiting example of Phase 3, a patient enters the clinicalpathway, and proceeds through the usual standards of diagnosis andpatient data collection, including patient history. This forms thepre-decision data. The patient is assigned a virtual patient ID in theIVPF. The clinician would select the appropriate module(s) relevant tothe disease in question and suitable for informing the clinical decisionat hand. The clinician would select one or more optimization criteria.Restrictions to the control parameters would be made at this time. Forexample, a clinician may exclude a particular type of therapy from theoptions of the module, for patient-specific reasons.

The module(s) will have certain input specifications, and these will bederived from the pre-decision data where known, and input into thesoftware application by the clinician. This input will immediately placethe real patient into a patient-specific virtual cohort with parametersin the same range as those of the patient. The IVPF will thenautomatically use the virtual patient database to determine the optimalvalues of the control parameters. As described earlier, these could beas simple as a binary decision, or as complicated as determining thesequence and dosing of a mix of several drugs.

The results will be presented to the clinician in an information paneldisplayed on a software application. A feature is that the interfacewill be interactive. The clinician can interrogate the results on manydifferent levels, to understand the implications of the various optimaltherapies that are being presented to them. By further varyingtherapeutic conditions and any risk-reward values, the clinician willhave a feel for how sensitive the predictions are for the particularpatient and the associated diagnostic and care-related factors.

The results presented on the interface may be statistical in nature,based on the selected optimization criteria. If appropriate to theclinical decision, several options can be compared to standard of care(SOC) results. The results will be variable depending on the settings ofone or more risk-reward sliders. These sliders control the sensitivityof the optimization algorithm to include the risk of predictive errordue to various clinical and algorithmic factors. These sliders mayinclude, but are not limited to, the risk of errors in therapeuticadministration; the risk of patient miscompliance with therapeuticregimen; the risk of drug toxicity; the risk of promoting existing orpotential co-morbidities; risk of errors in the measurement of patientdata; stochastic effects in the SM; the effect of highly variableoutcome landscapes in the SM output. Additional details are in thetechnical implementation section.

A feature of the present disclosure is the ability of the clinician tointeract with the results in real time through the setting oftherapeutic control restrictions and values of risk-reward weighting.This real-time analysis is performed using the VPD and the associatedanalysis tools described herein. Example user interfaces implementingthis feature are described below with reference to FIGS. 11 and 12.Since mathematical models take time to simulate, real-time analysis of agiven SM may not be possible if simulations have to be run for eachpatient at the time of diagnosis. Furthermore, real-time interactionwith the results also may not be possible without the framework of apopulated database that is analyzed with integrating tools. Thevariation of optimized predictions due to one or more clinician inputsdepends on the example framework and equivalents that are proposedherein.

When applicable, the IVPF will suggest that the measurement ofadditional patient data could lead to a more refined prediction. Forexample, if the patient is in a virtual cohort where treatment outcomesare sensitive to a particular molecular expression that has not beenmeasured in this particular real patient, then measurement of thismarker in histological sections could lead to improved predictions fromthe IVPF. The clinician would then decide whether or not to measure theadditional data, if possible, for a subsequent reanalysis of theclinical decision.

Once the clinician receives the results from the IVPF softwareinterface, they would make a final decision on the treatment strategy.This actual decision would then be input into the IVPF, and the patiententers into Phase 4 (140).

FIG. 4 illustrates there is illustrated a framework for Phase 4 (140),where prospective patient tracking and dynamic therapy optimization isperformed. Phase 4 of the IVPF will serve as a patient-specific trackingand prediction system, delivering dynamically optimized therapyrecommendations for each patient on an individual basis. Unlike currenttools which use only an instantaneous snapshot of the patient to derivea single prediction, this framework explicitly uses temporal patientdata to refine therapeutic predictions and minimize the risk oftreatment failure. If there are any variations in the protocol oftherapy chosen at diagnosis, e.g. a patient misses a dose, or changestheir appointment, this information can be input to the IVPF for animmediate analysis of the implications for optimal therapy, based on theinformation contained in the VPD. The risk-reward analysis will providea new assessment of the risk for any particular negative event, andfurthermore therapeutic changes to improve the risk-reward balance maybe suggested by the framework.

For example, in Phase 4, once the treatment decision has been chosen inPhase 3, the IVPF calls on the math module 106 to perform simulations offuture outcomes under this therapy for the patient-specific virtualcohort 142. The temporal data from these simulations are stored in theVP database 146 so that it can be directly compared with real datagathered from the patient, either at the next follow-up visit or fromremote patient reporting.

When new patient data are available, the additional data 150 collectedfrom the patient are input into the IVPF app 126. By comparing thesedata with stored temporal simulation data, the patient-specific virtualcohort can be further refined (at 148) to exclude those areas of thecohort that do not match the true progression of the patient. Theintegration and optimization described in Phase 3 is used (at 152) todeliver new optimal treatment strategies 154 with this refined VPcohort. These updated recommendations are returned to the clinical userin order to inform the choice of follow-up treatment. Further refinementof the risk-reward (RR) sliders, based on objective clinical observationof the patient response to date, can be performed by the clinician atthis point. The clinician would then make a decision on the continuingcourse of therapy, which may be to remain on the original therapeuticregimen, or modify in accordance with new predictions. Once thefollow-up therapy is chosen, this may again be input into the IVPF togenerate new temporal data. Phase 4 may be repeated as necessary foreach follow-up visit until the care has been completed.

The virtual patient database generated from the simulation model will begreatly enhanced over time as patient specific data is generated in theclinic and used to both populate the VPD and validate specific results.In other words, the actual data gathered from patients can be used tocontinually refine the weighting algorithm across parameters andvariables that were previously unmeasured in historical datasets. Thisfeed-forward approach allows for better predictions to be made forsubsequent patients entering the system. The trajectory of each patientspecific virtual cohort within the greater space of all virtual patientscan be used to analyze the biological factors prevalent in the disease,therefore shaping likelihood distributions for unmeasured/unmeasurableparameters. For example, an unmeasurable patient parameter such asmicrometastatic burden might eventually be calculated as a likelydistribution by the IVPF by analyzing the possible burdens associatedwith previous patients, as determined by the refinement of VP cohortsand associated outcomes.

This process of algorithm improvement will be accomplished byimplementing a machine learning environment, where the algorithms usedto deliver optimal strategy will be analyzed to compare virtual patientweighting distributions and actual patient distributions. Thiscomparison can lead to adjustments of the weighting algorithms, if thereis a discrepancy between the real and assumed distributions. A similarprocess could be used to refine the effects of therapy as determined bythe SM. Machine learning can check for skewed results that areconsistently offset from the true results, suggesting weightingimbalances in the optimization and risk-reward algorithms.

Example Environment

FIG. 5 is a schematic block diagram of the components of the IVPFenvironment 500. The IVPF may include a processing core 504, databaseservers 502, and a clinical device 506 running the interface application126 to implement the four phases described above. The implementationIVPF within the environment 500 may operate in four layers. The first,innermost layer is a disease-specific simulation module. This may bedeveloped for specific diseases by biologists, clinicians,mathematicians and/or statisticians to simulate a particular aspect ofthe disease. Examples may include a model of tumor growth, a model ofdrug pharmacokinetics and diffusion into the disease site, etc.

The second layer is the virtual patient database 122 within the databaseservers 502. The database 122 may be divided into two main sections:standardized outcome data and temporal data. An optimized outcomedatabase is a collection of optimal outcomes produced by using thesimulation modules, encompassing the broad spectrum of possible patientsand treatments relevant to the module in question. The temporalsimulation database is where patient-specific simulations for specifictreatment strategies are stored for use with follow-up data from eachpatient using the system.

The third layer is the simulation database integrator and optimizer. Theintegrator will take patient-specific data to combine the resultscontained within the virtual patient database, producing resultsrelevant to a patient-specific virtual cohort, which is smaller than theentire virtual cohort. Additionally, the integrator can use temporalresults from patient follow-up data to further refine thepatient-specific virtual cohort. The optimizer uses the patient-specificsubset of data to determine the optimal decision based on therestrictions of control parameters and other clinical considerations.

The fourth layer is the clinical interface application. This is softwarethat allows the clinical user to select the modules, input initial andfollow-up patient-specific data, restrict the treatment and optimizationcriteria, set risk-reward values, and view the results of the IVPFpredictions.

Simulation Modules

Below is a more detailed discussion of the simulation modulesimplemented within each of the layers above. In layer 1, the simulationmodules may have a specific format for usability in the other layers ofthe IVPF. First, they may accept as inputs two classes of data. Oneclass of input data is patient-specific biological measurements, denotedI. The second class of data is clinically-adjustable control parameters,denoted R. Both forms of inputs may only be permitted within anacceptable domain, defined by the simulation module. With a givendefinition of inputs, (I, R), the module then exports one or moreoptimization metrics. The optimization metrics are informative of eachdesired optimization criteria as derived from clinical practice. In thisframework, the modules act as functions of I and R and return theoptimization metric(s).

Each module may specify the following:

-   -   Input parameters (I):        -   Domain: Each input parameter is assigned a biologically            permissible domain. The domain is bounded and can be            discrete or continuous. Possible examples:            -   Number of cells at time of therapy: A discrete parameter                with integer values between 1 and 10{circumflex over                ( )}12 inclusive            -   Age: A continuous variable between 0 and 125 years            -   Sex: A discrete variable with two options (i.e., 0 and                1)            -   Biomarker expression: a continuous variable with range                0% to 100%            -   Production rate of a cytokine: a continuous variable                from 0 to 1.3 mM/day        -   Distribution: Each parameter domain is accompanied by a            probability distribution function (PDF). This describes the            expected values of the parameter. The distribution is used            for sampling the domain of the parameter when a precise            measurement is not known. The default PDF is linear over the            domain.        -   Input parameters need not be measured or even measurable at            the time of module development    -   Control parameters (R):        -   Each clinical parameter is directly derived from a            controllable clinical therapeutic variable.        -   Domain: The domain of clinical control parameters is            identified and bounded    -   Module outputs        -   Optimization metrics: these output data are the results that            will be used by the integrator and optimizer for deriving            virtual patient cohort statistics. The output can be a            continuous metric, or a discrete outcome. Examples:            -   Remission time            -   Toxicity measure            -   Cured/not cured            -   High, medium, low risk        -   Domain error code: indicates that the generated input call            is outside of the bounds of the model's use. This is for            cases where the input domains are dependent on each other.            This flag will tell the database to ignore these results.

In layer 2, the VPD may be split into two datasets: (1) the optimizedoutcome database, and (2) the patient-specific temporal simulationdatabase. Though both databases operate in the same multi-dimensionalparameter space defined by the particular mathematical module, themethods of populating the databases are different because of thedistinct clinical needs of Phases 3 and 4.

The Optimized Outcome Database

The optimized outcome database, a subset of the VPD, is generated sothat it will be useful to any possible patient that enters the clinicfor the first time. Therefore the database has to cover the entire spaceof parameters and therapy options. Since complete analysis of the entirespace each time a new patient enters the system is prohibitive, weinstead propose a sparse but intelligently-generated optimized outcomesdatabase so that the space can be reconstructed rapidly enough todeliver a real-time recommendation for a specific patient. The databasemay, for example, be populated by a combination of a genetic algorithmand variable-step-size iterative method. Since the dimensionality ofinputs accepted in a simulation module can be very high, the approach ofusing fine-grained simulation of all points in a discretizedinput-parameter space is likely to be prohibitive both in terms of datastorage and the time needed to simulate such a system. Therefore, anadaptive-step-size approach may be chosen. The goal of the databasegenerator is to establish the locations of local optima and gradientstrengths along each dimension of input data. As more simulations arerun with the module, the database would continue to accumulate points inthe range of outputs, lending more detail to the landscape of eachoptimization metric.

For a given module, Layer 2 will generate an outcome database. DuringPhase 2, the outcome database will be populated across the fullpermissible range of input and output parameters, so that the clinicaltool in Phase 3 need only query previously run simulations to findoutcomes for optimization relative to patient-specific data.

Two main processes can be used to populate the database:

-   -   Coarse-grained simulations across the grid of input and control        parameters        -   This approach gives a sampling of the range of the module            output        -   The step size will be variable in each dimension, and            dependent on the gradient of the output metric        -   The goal is to characterize not only areas of good and bad            metric values, but also to find areas where the slope may be            high. High slope of the output metric corresponds to higher            risk in giving treatments within that range of control            parameters    -   A genetic algorithm (GA) to find the optimal control parameters        R¬opt in the space of I        -   This second approach will seek optimal therapy within the            space of I using a genetic algorithm. The process will            generate a list of sequentially less optimal control            parameters R¬opt in each hyperplane of I. These serve as the            foundation for additional simulations in the area of the            optimum in order to find the risk gradient associated with            the optima        -   The GA will use mutation and recombination of the control            parameters to converge on local minima        -   Gaussian exclusion will be used to find subsequent minima in            the space, until the required number of minima have been            found

All simulations may be stored in a managed database that is able to berestricted to any range of input and control parameters. These processesoccur independently for each output metric supported by the module. Thecomplexity of the model will dictate the necessary simulation resolutionachievable in such a database.

The Temporal Simulation Database

The temporal simulation database, a subset of the VPD, containstime-course data generated by simulations for a specific patient. Whenthe initial patient therapy is decided at the end of Phase 3, thisinformation fixes the control parameters for the patient. The IVPF willthen use the mathematical module to generate simulations that predictthe time-course of patients contained within the patient-specificvirtual cohort subject to the administration of the actual therapydecided by the clinician. In this case, the algorithm will start with acoarse-grained sampling of the cohort parameter space, and then continueto add finer sampling until the patient returns for follow-up diagnosis.The simulation data is stored with a temporal resolution that would berelevant to typical follow-up times. In other words, a disease where thefollow-up times are spaced apart by 6-12 months would not need atemporal resolution of days, whereas a fast-progressing disease thatrequires weekly monitoring may require temporal resolution on the orderof one day or less. These criteria are module-specific and would bedetermined in the development of the module.

When the temporal simulation database is populated, correspondingoutcomes are stored in the optimized outcome database. This will permitdynamically optimized therapy decisions to be rapidly made duringpatient follow-up.

In layer 3, the database integrator may use the virtual patient outcomedatabase to generate a subset cohort of virtual patients. This cohort isgenerated through the input of data (P) from a single clinical patient,entered through the clinical interface application. Thispatient-specific data P will restrict the multi-dimensional domain ofthe set of parameters I, and generate a correspondingly smaller subsetof outcome data (the patient-specific virtual cohort). This derivationwill include an interpolation algorithm on the dimensions of R followedby an integration algorithm across the dimensions of P, with thepossible use of weighting if applicable. Finally, the integrated data issmoothed according to the risk-reward inputs provided by the user todetermine a suitable set of optimal recommendations for the specificpatient, based on the individual patient data which has been input.

The interpolation algorithm will take the optimum data points stored inthe simulation database and construct a function (g(P,R)) composed ofmultiple Gaussian curves with heights corresponding to the value of theoptimization metric at each position in R corresponding to an optimum.Each point in the restricted domain of P with existing simulation datawill have such a function. The integration algorithm will then combinethese functions with the appropriate weighting function for eachparameter value in P. In other words, the Gaussian functions g will bemultiplied by the weights attached to the space P and then summed. Thisproduces the patient-specific outcome function, which incorporates theuncertainty in P across the effects of control parameters R. Once thisfunction is generated, it is smoothed by the selected values of therisk-reward sliders, such that lower values of risk-reward correspond togreater smoothing of the outcomes across the dimensions relevant to theparticular risk being calculated. This smoothed function is analyzed todetermine the maxima, and these maxima are ranked to form the basis ofthe recommendations for control parameters R_opt that are returned tothe user.

The optimization process is illustrated in FIGS. 6, 7, and 8 by using asimplified module and framework algorithm to simulate the process. Themodule used here simulates tumor growth under the application of atreatment that is controlled by a dose fraction parameter, labeled R1.Additionally, the model contains a single patient-specific parameter,labeled I1. FIG. 6 represents a high-resolution output of the clinicaloutcome (O1) predicted by the module, with I1 on the vertical axis anddose fraction R1 on the horizontal axis. The color output shows thepatient relapse outcome for any pair of I1 and R1, with white being thebest patient response and black being worst. This model predicts thatfor any given patient-derived parameter value of I1, there are twochoices of R1 that the clinician can use to maximize the positiveclinical outcome. This optimal value of R1 depends on thepatient-derived value of I1. For a specific value of I1 (0.85 in thisexample), a cross section of the output is shown in FIG. 7, with bestoutcomes being positive. The optimal selection for the dose fractionparameter R1 is about R1=0.04, with a secondary maximum at R1=0.64.

FIGS. 6 and 7 represent the ideal situation where a very fine grid ofthe entire parameter space {I by R} can be explored. Since it iscomputationally prohibitive to simulate a complex multidimensional modelin this resolution, the IVPF will instead find the optimal controlvalues for a series of input parameter values, as described earlier.This information can then be used to derive an outcome function that isthe integration of the range of patient-derived input (P), smoothed bythe value of a risk-reward slider that mitigates risk for poortherapeutic outcome. For example, if the user inputs a patient-derivedrange for I1 between 0.7 and 1.0 in the above example, and selectsmoderate risk-reward, the integration and smoothing algorithm willproduce the output shown in FIG. 8, again with best outcomes beingpositive. The optimum value of R1 has shifted to 0.72, and this is thevalue that will be the primary recommendation to the clinician. Thesecondary maximum corresponds to R1=0.11. This can also be given as anoption, with a comparison of outcomes for both choices of R1.

The value of the risk-reward slider is best understood by consideringthe detailed output of FIG. 7. Based on the output of this particularSM, the most successful treatment dose is adjacent to a very steep slopein outcome. However, it would be risky to aim for this dose, since aslight error in dosing would change the outcome from very good to verypoor. In other words, there is a high risk to choosing the true optimaltherapy, in that very poor outcomes are likely if there is slight error.Upon examining the outcome generated in FIG. 8 after applying moderaterisk reward, it is clear that the recommended dose is higher than thetrue optimum, precisely to minimize the risk that slight errors indosing will produce drastically different results.

In this example, the risk-reward setting has shifted the optimumrecommendations for R1 from about 0.64 to 0.72 and from 0.04 to 0.11. Inaddition, the best outcome prediction for the two possible therapeuticrecommendations has swapped, so that the right peak is more likely tobenefit the patient on average. This is because the left peak, whilepotentially producing a more successful result, has more risk of pooroutcome due to uncertainties in therapeutic regimen and patientparameters.

In Phase 4, there is an additional method for refining thepatient-specific virtual cohort. By using temporal data generated in theperiod of time between a patient's initial therapy and subsequentfollow-ups, the IVPF can check the predictions made for each simulationin the patient-specific virtual cohort. Armed with temporal follow-updata, the IVPF will discard outcomes of simulations that are notvalidated by the temporal data. This temporal validation will likelyrestrict the patient-specific virtual cohort to a smaller, more targetedpopulation, leading to better predictions. From a technical perspective,the algorithm will weigh the outcomes from the temporal simulationsaccording to their temporal fit with the true patient data. Theoptimization routine will therefore be weighted towards thosesimulations that best tracked the actual patient progression.

FIG. 9 shows a schematic of how the databases are used in combinationwith the patient-specific virtual cohorts to determine dynamicallyoptimized treatment strategies. The outcome database (scattered dots ineach panel) is populated to cover the entire space of possible patientdata. When a patient enters the system, their patient-specific datadefines the patient-specific virtual cohort (gray rectangle of panel(a)), determining a subspace of optimization. Layer 3 produces anoptimized therapy for the patient (large dot in panel (a)). Immediately,Phase 4 commences. Temporal data for the patient-specific virtual cohortis generated (organized series of dots in panel (b)) and stored. Whenthe patient returns for follow-up, the newly collected patient data iscompared to the predictions of the temporal database. Simulations areweighted based on how well they predicted the patient progression,leading to a refined patient-specific virtual cohort (lighter area ofthe PSVC in panel (c)). This new cohort is optimized for therapy,leading to a new treatment prediction (large dot in panel (c)). Theprocess repeats with each follow-up visit, so that therapyrecommendations are adapted based on each new collection of data fromthe patient.

An implementation of Phase 4 with a SM that uses two patient parametersand two therapy control parameters is shown in FIG. 10, using as anillustration an extended example of the predictive mathematical moduledescribed above for the example of Phase 1 and 2. In this extendedmodule, there are two patient-specific parameters, p1 and p2. Theparameter p1 represents the ER staining in the biopsy tissue, and p2represents the percentage of Ki 67 staining. For this sample module,there are two control parameters (r1 and r2) that adjust the deliveryregimen of a chemotherapeutic drug in combination with hormone therapy.Control parameter r1 is the dose fractionation, and r2 is the deliveryinterval. The module output is the tumor burden at one yearpost-therapy. As in the previous example, all inputs and outputs arenormalized to the range of [0,1].

FIG. 10 particularly illustrates the process used by the IVPF in Phase 2and Phase 3. Panel (a) shows a sample VPD generated using the moduledescribed above. The sampling space has a resolution of 0.1 across bothdimensions of the parameter space, with values of p1 and p2 shown on thetop and right axes of the overall grid. At each sample point, there is aheat map of the reconstructed outcomes over the space {r1×r2} with axesas shown on the lower left map. These heat maps were generated from thedata points stored in the virtual patient database (VPD) for eachsampling point. The inset shows an enlargement of one heat map for theparameter combination (p1=0.8, p2=0.2) with white areas representing thebest outcome and dark areas representing poor outcomes. Superimposed aredata points representing the stored VPD data generated by the IVPF inPhase 1.

Suppose that the initial diagnosis for a particular patient found thatthe level of ER staining (p1) was between 0.6 and 1.0 (in normalizedunits), and that Ki-67 stain (p2) was at most 0.8. These bounds would beentered into the clinical application and the system would place thepatient into the initial diagnosis PSVC shown in the larger of the twooutlined rectangles of FIG. 10. The system will then integrate theoutcome data in this PSVC to derive the cohort outcome function usingthe database integration techniques described in the Phase 1. When theintegrated outcome function has been derived, this information isreturned to the software application to generate an appropriateprediction of the outcome.

Clinical risk-reward adjustment. Once the cohort outcome function iscalculated, the clinician can interact with the suggested outcomes byadjusting a risk-reward (RR) slider. The purpose of this particularclinical adjustment parameter is to inform the clinician about theconfidence of the derived predictions and their sensitivity to variancein the measured patient and therapeutic parameters. When the risk-rewardslider is set to high-risk high-reward, the optimization algorithm willfavor those therapies that have the best possible outcome out of alltherapeutic options, without consideration of the sensitivity of thisoutcome to variations in parameter values. When the slider is set tolow-risk low-reward, the optimization algorithm will find the besttherapy that minimizes the risk of poor outcomes due to parametervariations. The implementation of the RR slider in this particular casecan be accomplished by using, for example, Gaussian smoothing across theparameter dimensions and then deriving the optimum treatment from thesmoothed outcome function. There can be multiple risk-reward sliders tocover different clinical contingencies. For example, drug efficacy, drugtoxicity, patient compliance, and impact of other co-morbidities canhave risk-reward sliders that interact.

This output generated by this RR process is illustrated in FIG. 10,panels (b1-b4). The integrated outcome data from the initial PSVC fromFIG. 10, panel(a) are shown for four values of the RR slider. Panel (b1)shows the high-risk high-reward setting. Here, the algorithm hasselected the best overall therapeutic option for the PSVC, leading to acohort-wide average outcome of 0.34. The suggested optimal combinationof treatment parameters (r1=0.45, r2=0.41) is shown by the open circle.However, the region of very poor outcomes (dark area) on the left sideof the heat map suggests that there is some risk of a bad result fromtherapy if there is some variation in the true parameters. As the RRvalue is lowered in subsequent panels, the recommended therapy travelsalong the line, away from the area of poor outcome. The correspondingcohort average outcome values decrease, as do the associated risks. Thelow-risk therapy suggestion (r1=0.64, r2=0.3) in panel (b4) will have alower chance of a good outcome, but also a lower chance of a pooroutcome. In this particular example, the asymmetry of the outcomelandscape leads to changes in optimal therapy recommendation as afunction of the RR value. Other models that have more symmetric outcomelandscapes may see very little shift in recommended therapy. Havingexplored the RR settings, the clinician would be able to use the resultsfrom the IVPF to inform the actual therapy delivered to the patient.Once a treatment course is decided, this selected therapy would be inputinto the interface application for use in the Phase 3.

Phase 3 implementation. When a clinician inputs the chosen therapy atthe end of Phase 2, the IVPF will call on the mathematical module togenerate patient-specific temporal data for later comparison with actualpatient follow-up data. The IVPF will fix the treatment parameters (e.g.r1, r2) to those that were selected for the patient. The system willthen call on the mathematical module to simulate temporal data across asampling space of the initial PSVC (large rectangular outlined area ofFIG. 10). This data will be stored in the VPD, with a temporalresolution appropriate to the follow-up conventions of the particulardisease. For example, a disease with expected follow-up frequency on theorder of one year will not need the same temporal resolution as one thatis managed on a weekly basis. Since the return date of the patient maynot be precisely specified at the time of initial therapy, the temporaldatabase will store a time series of all variables and outcome metricsin the module for each sample in the PSVC. The resolution of thesampling space of the PSVC will be determined by the computational poweravailable to simulate the temporal database in quasi-real time. Sincethe temporal data will be used only at the time of follow-up, there isno need to simulate far beyond the real elapsed time since patienttherapy began. In other words, if it has been 50 days since the patientbegan treatment, all simulations in the temporal DB for that patientwill have been simulated out to 50 days, plus some cushion. Thismaximizes computational efficiency and also gives the highest samplingresolution subject to computational power.

At the time of follow-up, new data will be collected from the patient.The clinician would return to the interface app, enter the virtualpatient ID, and then input the appropriate follow-up data. The IVPF willcompare this patient data with the simulated temporal data evaluated atthe actual follow-up time. For example, if a patient returns after 60days, then the simulation outcomes are queried for t=60 within thetemporal database. The comparison of simulated and patient data willgenerate a weight for each parameterization in the sampling space of thePSVC. Some simulations will match well, and these will be assigned ahigher weight. Simulations that poorly predicted the follow-up data willhave a lower weight. Once this weighting is determined, the IVPF willthen refine the PSVC by including these weights in the follow-uprecommendations. Using the example above, suppose that the simulationsin the range of (0.8<p1<0.9, 0.1<p2<0.3) were well matched with theactual follow-up data, while simulations outside of this range were poorpredictors of progression. Then the IVPF would effectively define a newrefined PSVC through a weighting function that gave weight only to thesimulations in that range. This new PSVC is indicated by the smalleroutlined rectangle of FIG. 10, panel (a). For clarity here we have givenfull weight to these simulations and no weight to simulations outside ofthat range. In practice, the entire range will have continuous weightingapplied to it.

The use of this weighting will be included in the data integrationprocess (as described in Phase 1 and Phase 2) to derive a new predictionof follow-up therapy. FIG. 10, panels (c1-c4) shows the outcomes forfour values of the RR slider for the weighted outcomes from the refinedPSVC. Again, the therapy recommendation changes with different RR. Ofinterest is that the expected outcomes have improved compared to theinitial therapy recommendations of FIG. 10, panels (b1-b4). For thehigh-risk setting, the average cohort outcome has increased from 0.34 to0.54. This is due to the fact that the temporal data fitting hasnarrowed the size of the effective PSVC so that new predictions can bebetter tailored to the patient. The Phase 3 can be repeated as necessarywith each patient follow-up visit. Increased data collectionsubsequently leads to more personalized, dynamically optimized treatmentin the clinic.

In Layer 4, the clinical interface Application 126 may be a softwareapplication (app) is a multi-platform tool that allows a clinician tointerface with the IVPF, using the system to get personalized resultsfor an individual patient. Designed to use minimal resources locally(calling pre-stored information remotely) and therefore capable ofrunning on almost any mobile device e.g. Tablet computer or smart phone.The front end of the app, shown in FIG. 11, will be where the clinicianchooses the modules specific to the disease as well as the optimizationcriteria. In FIG. 11, 1101 is a patient gender and disease siteselection, 1102 is a metastatic site selection, 1103 are module specificoutput options, 1104 is a selector for a choice of historic Databasesfor validation purposes and 1105 is a touch sensitive interface allowsdirect choice of primary disease site and metastatic sites.

The clinical interface, shown in FIG. 12, is where the clinician inputsthe patient-specific data, therapeutic restrictions, and furtheroptimization criteria. In FIG. 12, 1201 is a patient data input, whereinmulti-level drop downs ties to specific disease site, Reference 1202 aredisease specific therapy options, 1203 are optimization criteria, 1204is one or more risk-reward sliders allowing the clinician to weigh thetrade off between predicted/actual therapeutic success due touncertainties in patient care. Reference 1205 shows therapeuticoptimization results, where the left panel shows range of treatmentoptions and relative outcomes and the right panel shows a larger versionof the most successful strategy. Reference 1206 is a module outputselection, where different predicted outcomes can be visualized.Reference 1207 is a module specific output—visualization of outcomesboth historic and predicted may be shown.

The inputs from the interface are sent to the IVPF, which will quicklyanalyze the data from the VP database, subject to the constraints inputby the clinical user. The results from the IVPF are then displayed here,and adjustment of the clinical risk-reward slider(s) will shift theoutputs appropriately. The clinician would be able to page through allassociated outcome data from the simulated results.

EXAMPLE APPLICATIONS

Briefly outlined below are two examples of how the framework may beapplied to specific diseases in the clinic. These examples are notlimiting, since the framework is broadly applicable to a range ofproblems, but rather serves as an illustration of actual applications.Any decision system that can be quantified by an optimization metric andparameterized by measurable inputs would function within the framework.

Example (i): Risk Prediction in Large Granular Lymphocytic Leukemia(LGLL)

In LGLL, patients would benefit from the ability to estimate theseverity of progression of the disease after diagnosis. At present, theapproach used is “watch and wait,” in which the clinician will waituntil the disease begins to rapidly progress before giving treatment.However, this is often not the optimal time for therapy, it beingadministered too late. Being able to track and model patients in theclinic so that the onset of aggressive disease can be predicted wouldallow preemptive therapy to be given before the disease progresses toofar.

In order to use the IVPF framework, first a mathematical model of LGLLwould be developed. This could include various disease relevantpatient-specific inputs, such as blood cell counts and other bloodbiopsy measurements; ex-vivo cell culture experiment results providingdynamic information on T-cell replication rates; bone marrow biopsies tomeasure fibrosis; etc. The clinical control parameters could initiallybe limited to a binary decision of whether to treat or not treat. Theoptimization criteria would be some clinically relevant measurement ofdiseased clonal T-cells, perhaps combined with metrics of other symptomssuch as cytopenia.

Once the module was developed, it would go through the four phases ofthe IVPF:

Phase 1: The model would be validated against LGLL patient-databases, ofwhich several exist in the United States. Proceed to next phase oncevalidated.

Phase 2: The outcome database would be generated.

Phase 3: Would begin to aggregate patient data with implementation intothe clinic. The outcome data would be a prediction of risk ofaggressiveness without therapy. Using this output for a given patient,the decision to treat or wait would be made by the clinician. I.e.,patients with low risk for aggressive disease would be placed on “watchand wait,” while those that the IVPF predicted high aggressiveness wouldreceive therapy at once.

Phase 4: Subsequent visits by the patients on the “watch and wait” planwould generate new blood biopsies which would be analyzed for patientprogression. These new data would be used to refine the subset ofprogression simulations that the patient satisfied. This would lead to anew metric of aggressiveness. In particular, the IVPF would be able toindicate which patients that were on the “watch and wait” plan werebecoming more aggressive (i.e., time to treat) and which remainedindolent (continue to “watch and wait”).

Example (ii): Optimize Adjuvant Therapy for Breast Cancer Patientswithout Known Metastases

Many patients with primary tumors of the breast do not have detectablemetastases at the time of diagnosis and initial therapy. However, asubset of these patients do relapse with distal metastases after someperiod, even with application of adjuvant therapies post-surgery. Apressing question in the clinic is what is the best type of adjuvanttherapy to administer for patients that have no distal metastases oninitial scans. There are various hormonal therapies, chemotherapy,radiation, and targeted therapies, all of which can be combined invarious ways. Without any residual disease detectable, there is no wayto optimize therapy based on metastatic biopsies.

To use the IVPF framework to address this question, a model ofmetastatic growth of breast cancer cells in various distal sites (bone,brain, lungs) would be developed. The models would simulate the effectsof various clinically relevant treatments. Relevant parameters would beprincipally derived from the primary tumor, including the status ofhormones, metabolic and growth markers, and other relevant molecularproperties. Toxicity would be part of the model. Clinical controlparameters would be the selection and durations of therapies.Optimization criteria would be the minimization of potential metastaticgrowth.

Once the module was developed, it would go through the four phases ofthe IVPF, as follows:

Phase 1: The model would be initially validated against the database ofbreast cancer patients, both with and without metastatic relapse. Thetherapies would be SOC, and outcomes would have to match the historicalrecord. Proceed to next phase once validated.

Phase 2: The outcome database would be generated.

Phase 3: Patients initially diagnosed with primary breast cancer wouldhave their biopsies analyzed to produce patient-specific data. The IVPFwould process this data to find an optimal therapy recommendation thatwould minimize the chance of metastatic recurrence without causingundesired toxicity.

Phase 4: Subsequent visits by the patients would include scans formetastatic cancer. In addition, any relevant physiological measurements,for example hormonal levels and toxicity responses to the drugs, couldbe used to check model predictions. Patients that scanned clean wouldhave new temporal data on toxicity symptoms that could lead to therapyadjustments.

Other Applications

With some modifications to the clinical interface application, the IVPFcould be used with any disease where predictions of risk and outcomesare valuable in determining a course of action for the patient. Thiswould not be limited to cancer; indeed it is hard to imagine a diseasewhere patient-data would not be useful for predicting outcome. The IVPFcan operate on any timescale, so acute infections lasting a matter ofdays are as tractable as chronic diseases that persist for decades. Dueto the modular nature of the framework, any mathematical model thatsatisfies the conditions of input and output data can be used.Therefore, the IVPF could be used for problems outside of the biomedicalfield as well, although some changes to the interface app might have tobe made to match the specific needs of the field in question.

It is possible to only measure a limited amount of biology for any givenpatient and it is impossible to simulate a true representation of aspecific patient—the VPD resolves these issues by using a hybridapproach that represents a single real patient with a cloud (cohort) ofsimilar patients. The accuracy of this cohort will improve significantlythe more the VPD is enriched, with patient specific virtual cohorts,refined by true temporal data gathered from individual patients. At thispoint analysis of the virtual cohorts for a given disease will revealnovel aspects of the disease that can only be obtained through our IVPFapproach. Specifically, this analysis may lead to new diagnostictechniques, new therapeutic strategies, novel biological associationsand mechanistic interactions. Furthermore, such analysis also appliesacross different VPDs and may indicate additionally novel commonalities.

Furthermore, the database and analysis tools generated in the process ofusing the system in a clinical setting are a valuable resource for usein subsequent clinical trials. The IVPF can be used to design virtualclinical trials, in which millions of virtual patients can be tested forkey diagnostic markers, toxicity, and efficacy of existing and novelcompounds. With an appropriately modified SM to address the noveltherapeutic approaches to be investigated by the Phase I trial, the IVPFcan run a Phase “i” trial. These results could assist trial designers incohort selection, therapy regimen strategies, and also predict thepotential risks faced by administration of the trial. The power of thisapproach would be extended by the use of a validated VPD that had beenrefined by machine-learning algorithms during the acquisition of realpatient data.

Decision Support Via Evolutionary Model Analysis

The framework of the present disclosure may be used to provide decisionsupport that evolves in conjunction with the increasing data collectedon a given patient. Patient data occurs at varying resolutions. Forexample, at the beginning of their clinical trajectory, the informationis sparse, often a single time point at diagnosis. As the patient istreated, additional timepoints are received. These data can be acquiredbased on standards of care, or through specific decisions from theoncologist such that the use of patient data changes through a course oftherapy.

With reference to FIG. 13, patients with single time point data (leftedge of figure) may be limited to support using statistical predictiontools derived from historical cohorts of similar patients, where similaris defined by one or more biomarkers or demographic information. Laterin the patient's treatment, there may be a few temporal data points(central section of figure) that are feasibly modeled with simplemathematical models that can constrain the possible dynamics of thepatient. Patients with longer histories or denser biomarker measurements(towards the right edge of the figure) benefit from a more detailedpredictive model, since different types of data are integrated andbetter fitting of the data is possible.

Thus, the framework uses multiple models, ranging from statistical tomechanistic, and evolves the predictive decision support based on newdata as it is collected from the patient. Two or more models, includingbut not limited to statistical, machine learning, AI, mechanistic, andhybrid models may be used. Weights for each model are assigned dependingon uncertainties arising from data fitting and model properties, e.g.models with better fits will have a higher weight.

As new data are entered into the patient record (e.g., the dots in FIG.13), the models are recalibrated and the weights are adjusted, leadingto updated decision support information (connecting arrows betweenmodels and Rx decision support). The framework also may suggest thebenefit of additional follow-up data collection events, in essenceoptimizing the data collection (with constraints) as well as how itgenerates predictions.

Patient Education and Empowerment Application

A direct benefit of the framework is that it can be used to presentscenarios to patients in a way that informs them of treatment outcomes,given various strategies. This can be done via, e.g., an app thatdescribes the range of outcomes and risk-reward considerations for eachtreatment strategy that is under consideration. Patients may exploredifferent dosing and schedule changes to see potential impacts to theirdisease course. This has the added benefit of educating the patient asto the impact of a specific treatment strategy as well as empoweringthem to suggest specific strategies for their care. It would also serveas a real time update of a patients progress on a given treatmentprotocol.

Integration of Personal Health Data

With poor temporal density of follow-up visits being an issue inoncology, collection of any additional data from patients can be usefulas a method to increase the accuracy of predictive models. Theappearance of symptoms that might indicate disease progression, ortreatment toxicities, would be reported by the patient and this can beadded to the collection of data for the patient, and enhancing theprediction of the decision support tool. This type of data could bedirectly entered via the interactive app that the patient used tomonitor their treatment and play out different treatment scenarios.

FIG. 14 illustrates and example workflow 1400, which begins with apatient being is diagnosed with a condition, e.g., cancer, at time 0(1402). At 1404, demographic information is collected, along withvarious diagnostic biomarkers (including but not limited to: imaging,blood, biopsies, physical exams, patient self-reporting, geneticanalysis, etc.), and this information is provided to the IVPF framework.

At 1406, multiple models are built for the particular disease that usedifferent aspects of the data that are collected to generate predictionsof how the patient may respond to the available therapy options at thestart of treatment. In general, these models will be statistical modelsas there are few temporal data points and these will have the highestweight at the start of the patient's course of therapy. At this stage,dynamic models, which generally require multiple time points to generatefitting constraints, will likely have lower weights at diagnosis. Theinitial data may restrict some of the parameters of the dynamic models,which allows for the IVPF to commence analysis of potential trajectoriesfor the patient, generating stored Virtual Patients in the VirtualPatient Database. This is done independently for each distinct modelused for the particular disease. The statistical predictions deliveredby each model are weighted depending on the uncertainty produced by eachmodel, and the choices of therapy are ranked and delivered to thetreating physician along with associated supporting information such as:the likelihood of successful outcome for each; the statisticaluncertainties in the predictions; trajectories of possible outcomes.

At 1408, after therapy commences, follow-up data is received that isadded to the IVPF framework. Typically, a point in time is reached wherethe patient may be eligible to alter their therapy or continue with thecurrent therapy. At this point, the patient's collected data is comparedagainst the large number of virtual patients that exist in the databaseof simulations, by collecting the same type of data from each virtualpatient (if available for that particular type of model) as wascollected from the patient themselves, at corresponding timepoints ineach simulation (1410). This is done for each of the one or more dynamicmodels used for the particular disease. For each model and itsassociated virtual patients, some simulations will match the change inthe collected biomarkers better than others, and the IVPF will weigh thevirtual patients for each model accordingly, to generate a cloud ofsimilar virtual patients and discard those that don't match theavailable data.

At 1412, a set of refined and weighted virtual patients for each modelis created by this process. These refined cohorts of virtual patientsare then subjected to “Phase i” trials (each set as a virtual patientcohort). For each model and virtual patient cohort, this virtual “Phasei” trial subjects the virtual cohort to each available treatmentpossibility (e.g., continue current therapy, add an agent, have atreatment break, switch to an alternative agent or agents, stop thetherapy altogether, etc.). The framework associates an uncertainty tothe results based on the range of outcomes observed in the virtual trial(because each virtual patient reacts differently to the administeredtherapy). These predictions from the different virtual trials run usingeach different model are then integrated together using new weightingthat is based on the uncertainties generated from the virtual trials.The weight of each model-generated virtual trial changes independentlybased on the prediction uncertainty arising from the model fits andparameterizations. The evolution of these weights allows continuousrefinement of the IVPF predictions, and furthermore allow ‘unexpected’data to be accounted for by different models.

In an example application, many aspects of cancer growth are stochastic,and therefore can arise at unpredictable times. For example, a cancermutation may cause a significant change in tumor dynamics and/orresponse to therapy. In this case, one or more models may be developedto simulate the dynamics of the tumor pre-mutation, and one or moremodels may be developed for the dynamics post-mutation. As patient datais received, the data may fit the first class of models for a time andnot the second class; after some time and the acquisition of additionaldata, there may be a change in the dynamics that fits the second classof model(s) better, and less so the first class. The IVPF can adjustweights for these models based on how the models fit at each givendecision point, as well as directly collected data (e.g., if a biopsyconfirms the mutation). Therefore, the patient acquiring the mutationmay lead to the weights shifting from favoring one set of models tofavoring a different set. This evolution can then potentially alter theoutcome of the therapy decision support recommendations.

In another example application, some models may be developed for certainaspects of the disease rather than the disease as a whole; for example,metastasis to different sites in the body (e.g., bone, lungs, brain) mayrequire different models, and these different models may be weighteddepending on whether metastatic lesions are found in the particularsites from patient scans. Alternatively, or in combination with patientdata, the weighting can also be based on historical statistical dataregarding the likelihood of metastatic dissemination occurring indifferent sites. These likelihoods can change over time in thehistorical record, and this can alter the weights of the differentmodels over time. For example, the likelihood of seeing metastaticlesions in one site may decrease with time on some therapy; therefore,the model(s) that simulate the growth of lesions in one site will begiven successively less weight over time if the patient's data doesn'tshow evidence of lesions at the site.

At 1414, therapy choices are ranked based on the outcome of the IVPFanalysis and delivered to the treating physician. The ranking may bedone through an application where a physician may be presented theavailable therapies, the likelihood of successful outcome for each, thestatistical uncertainties in the predictions, and trajectories ofpossible outcomes, as examples. Other options may be presented. Thephysician may change certain aspects of the optimization, including butnot limited to: restricting or expanding the types of therapiesavailable adding or removing the considerations and constraintsregarding treatment selection such as drug toxicity, payer status,financial toxicity; altering the outcome metrics that are considered inthe optimization such as overall survival, progression-free survival,minimize toxicity, and/or combinations of these; altering therisk-reward level for one or more aspects of risk related to one or moretherapy options. Toxicity biomarkers, payer coverage, and financialaspects of therapy can be included in the decision support vector at anytime

In accordance with an aspect of the disclosure, steps 1408-1414 may berepeated, as necessary, for example, each time that new patient data iscollected. In this manner, the virtual patient database is continuouslyupdated and added to, along with the associated weights for each virtualpatient in each model. New patients are generated by prioritizing thebranches in the virtual patient cohort that best match the patient data(i.e. have higher weights), and minimizing simulations alongtrajectories that have low weights. This pre-simulation is a key aspectof being able to deliver decision support rapidly when new patient dataarrives.

FIG. 15 shows an exemplary computing environment in which exampleimplementations and aspects may be implemented. The computing systemenvironment is only one example of a suitable computing environment andis not intended to suggest any limitation as to the scope of use orfunctionality.

Numerous other general purpose or special purpose computing systemenvironments or configurations may be used. Examples of well-knowncomputing systems, environments, and/or configurations that may besuitable for use include, but are not limited to, personal computers(PCs), server computers, handheld or laptop devices, multiprocessorsystems, microprocessor-based systems, network PCs, minicomputers,mainframe computers, embedded systems, distributed computingenvironments that include any of the above systems or devices, and thelike.

Computer-executable instructions, such as program modules, beingexecuted by a computer may be used. Generally, program modules includeroutines, programs, objects, components, data structures, etc. thatperforms particular tasks or implement particular abstract data types.Distributed computing environments may be used where tasks are performedby remote processing devices that are linked through a communicationsnetwork or other data transmission medium. In a distributed computingenvironment, program modules and other data may be located in both localand remote computer storage media including memory storage devices.

An exemplary system for implementing aspects described herein includes acomputing device, such as computing device 1500. In its most basicconfiguration, computing device 1500 typically includes at least oneprocessing unit 1502 and memory 1504. Depending on the exactconfiguration and type of computing device, memory 1504 may be volatile(such as random access memory (RAM)), non-volatile (such as read-onlymemory (ROM), flash memory, etc.), or some combination of the two. Thismost basic configuration is illustrated in FIG. 15 by dashed line 1506.

Computing device 1500 may have additional features/functionality. Forexample, computing device 1500 may include additional storage (removableand/or non-removable) including, but not limited to, magnetic or opticaldisks or tape. Such additional storage is illustrated in FIG. 15 byremovable storage 1508 and non-removable storage 1510.

Computing device 1500 typically includes a variety of computer readablemedia. Computer readable media can be any available media that can beaccessed by device 1500 and include both volatile and non-volatilemedia, and removable and non-removable media.

Computer storage media include volatile and non-volatile, and removableand non-removable media implemented in any method or technology forstorage of information such as computer readable instructions, datastructures, program modules or other data. Memory 1504, removablestorage 1508, and non-removable storage 1510 are all examples ofcomputer storage media. Computer storage media include, but are notlimited to, RAM, ROM, electrically erasable program read-only memory(EEPROM), flash memory or other memory technology, CD-ROM, digitalversatile disks (DVD) or other optical storage, magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,or any other medium which can be used to store the desired informationand which can be accessed by computing device 1500. Any such computerstorage media may be part of computing device 1500.

Computing device 1500 may contain communications connection(s) 1512 thatallow the device to communicate with other devices. Computing device1500 may also have input device(s) 1514 such as a keyboard, mouse, pen,voice input device, touch input device, etc. Output device(s) 1516 suchas a display, speakers, printer, etc. may also be included. All thesedevices are well known in the art and need not be discussed at lengthhere.

It should be understood that the various techniques described herein maybe implemented in connection with hardware or software or, whereappropriate, with a combination of both. Thus, the processes andapparatus of the presently disclosed subject matter, or certain aspectsor portions thereof, may take the form of program code (i.e.,instructions) embodied in tangible media, such as floppy diskettes,CD-ROMs, hard drives, or any other machine-readable storage mediumwhere, when the program code is loaded into and executed by a machine,such as a computer, the machine becomes an apparatus for practicing thepresently disclosed subject matter.

Although exemplary implementations may refer to utilizing aspects of thepresently disclosed subject matter in the context of one or morestand-alone computer systems, the subject matter is not so limited, butrather may be implemented in connection with any computing environment,such as a network or distributed computing environment. Still further,aspects of the presently disclosed subject matter may be implemented inor across a plurality of processing chips or devices, and storage maysimilarly be affected across a plurality of devices. Such devices mightinclude PCs, network servers, and handheld devices, for example.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implem.

What is claimed:
 1. A model-based method for therapeutic decisionsupport in an Integrated Virtual Patient Framework (IVPF), comprising:defining a first disease-specific model, and one or more additionaldisease specific models that provide therapeutic decision support inaccordance with a temporal relationship with multiple timepoints for apatient diagnosed with a condition; determining, in accordance with eachset of patient data at given timepoints, a first therapeutic decision byapplying a first weight to the first disease-specific model, a secondweight to the second disease-specific model, and subsequent weights tothe additional disease-specific models; ranking therapy choices of thefirst therapeutic decision in a user interface; thereafter, repeating,for subsequent patient data received by the IVPF: adjusting the firstweight, second weight and additional weights in accordance with thetypes of subsequent patient data received; comparing the subsequentpatient data to virtual patent data in a database of simulated outcomesdetermined using the first disease-specific model, the seconddisease-specific model, and additional disease-specific models;determining, in accordance with the comparing, a subsequent therapeuticdecision by applying an adjusted first weight to the firstdisease-specific model, an adjusted second weight to the seconddisease-specific model, and additional adjusted weights to theadditional disease-specific models; and ranking subsequent therapychoices of the subsequent therapeutic decision in the user interface. 2.The method of claim 1, further comprising comparing simulated outcomesdetermined using the first disease-specific model, the seconddisease-specific model or the additional disease-specific models withhistorical outcomes for actual patients.
 3. The method of claim 1,further comprising adjusting the first weight, second weight andadditional weights in accordance with the types of subsequent patientdata received each time the subsequent patient data is received.
 4. Themethod of claim 3, wherein the adjusting provides for continuousrefinement of the subsequent therapeutic decision.
 5. The method ofclaim 1, wherein the first disease-specific model is a statisticalmodel, the second disease-specific model is a simple dynamic model, andthe third disease-specific model is a complex dynamic model.
 6. Themethod of claim 5, wherein the first therapeutic decision issubstantially determined in accordance with the statistical model. 7.The method of claim 5, wherein subsequent therapeutic decisions aresubstantially determined in accordance with a combination of the simpledynamic model and the complex dynamic model.
 8. The method of claim 1,further comprising determining an optimal outcome based on thesubsequent therapy choices.
 9. The method of claim 1, furthercomprising: receiving inputs in the user interface to alter aspects ofthe subsequent therapy choices; parsing the simulated outcomes in thedatabase; and comparing the subsequent patient data and the alteredaspects of the subsequent therapy choices to the virtual patent data inthe database of simulated outcomes; and updating the subsequenttherapeutic decision in accordance with the altered aspects.
 10. Themethod of claim 1, further comprising discarding simulated outcomes thatdo not match the subsequent patient data.
 11. The method of claim 1,further comprising determining at least one of the first therapeuticdecision and the subsequent therapeutic decision in accordance withpatent data that was acquired before the patient was diagnosed with thecondition.
 12. A model-based method for therapeutic decision support inan Integrated Virtual Patient Framework (IVPF), comprising: receiving,at a first time, first data associated with a patient; defining at leasttwo disease-specific models that provide the therapeutic decisionsupport in accordance with a temporal relationship with the first timeand a second time; determining an initial therapeutic decision byapplying weights to a first disease-specific model of the at least twodisease-specific models; receiving, at a second time later than thefirst time, second data associated with the patient; determining asubsequent therapeutic decision by: comparing the first data and thesecond data to virtual patients stored in a database of simulationsperformed in using the at least two disease-specific models; applyingweights to the virtual patients to generate a cloud of similar virtualpatients to the patient for each of the at least two disease-specificmodels; and subjecting the similar virtual patients to a trial processusing available treatments to determine a range of outcomes; and rankingsubsequent therapeutic decisions associated with the outcomes in a userinterface.
 13. The method of claim 12, further comprising comparing therange of outcomes with historical outcomes for actual patients.
 14. Themethod of claim 12, further comprising: receiving subsequent seconddata; and adjusting the weights of the virtual patients in response toreceiving the subsequent second data.
 15. The method of claim 14,wherein the adjusting provides for continuous refinement of the rankingof the subsequent therapeutic decisions.
 16. The method of claim 12,wherein the first disease-specific model is a statistical model, thesecond disease-specific model is a dynamic model.
 17. The method ofclaim 16, wherein the first therapeutic decision is substantiallydetermined in accordance with the statistical model.
 18. The method ofclaim 16, further comprising varying dynamic model from a simple dynamicmodel to a complex dynamic model as time progresses forward.
 19. Themethod of claim 18, wherein subsequent therapeutic decisions aresubstantially determined in accordance with a combination of the simpledynamic model and the complex dynamic model.
 20. The method of claim 19,wherein the simple dynamic model comprises one or more simple dynamicmodels, and wherein the complex dynamic model comprises one or morecomplex dynamic models.
 21. The method of claim 12, further comprising:receiving inputs in the user interface to alter aspects of thesubsequent therapy choices; comparing the subsequent patient data andthe altered aspects of the subsequent therapy choices to the virtualpatent data in the database; and updating the subsequent therapeuticdecision in accordance with the altered aspects.
 22. The method of claim12, further comprising presenting, for each of the subsequent therapychoices, a likelihood of a successful outcome, a statisticaluncertainty, and trajectory of possible outcomes.
 23. The method ofclaim 12, further comprising presenting scenarios the patient in theuser interface to informs the patient of treatment outcomes, givenvarious strategies.